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Abstract 

In order to understand the performance of a code under maximum-hkehhood (ML) 
decoding, one studies the codewords, in particular the minimal codewords, and their 
Hamming weights. In the context of linear programming (LP) decoding, one's attention 
needs to be shifted to the pseudo-codewords, in particular to the minimal pseudo- 
codewords, and their pseudo- weights. In this paper we investigate some families of codes 
that have good properties under LP decoding, namely certain families of low-density 
parity-check (LDPC) codes that are derived from projective and Euclidean planes: we 
study the structure of their minimal pseudo-codewords and give lower bounds on their 
pseudo- weight. 

Index terms — Linear programming decoding, message-passing iterative decoding, min- 
imal codewords, minimal pseudo-codewords, pseudo- weight, codes from projective planes, 
codes from Euclidean planes, pseudo- weight spectra. 

1 Introduction 

The motivation of this paper to look at minimal codewords and minimal pseudo-codewords 
is twofold. On the one hand we would like to give performance guarantees on linear pro- 
gramming (LP) decoding ^ [2], which is a decoding method that has recently emerged 
as an interesting approach to decoding codes. On the other hand, the connection made 
by Koetter and Vontobel [SI IS El between message-passing iterative (MPI) decoding and 
LP decoding suggests that results for LP decoding have immediate implications for MPI 
decoding. 

Apart from presenting some bounds on pseudo-weights that hold in general, this paper 
focuses solely on certain families of codes based on projective and Euclidean planes. One of 
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the reasons for this choice is that in the past, several groups of authors have experimentahy 
observed that such codes can perform very well under MPI decoding, see e.g. [01 El; and 
therefore these families of codes are a worthwhile object of study for making some first 
steps towards a rigorous understanding of the observed behavior. Another reason is that 
these families of codes have concise descriptions and large automorphism groups which may 
potentially be used to simplify their analysis. 

The rest of this paper is structured as follows. In Sec. I^J we will introduce the two 
main families of codes under investigation. After having reviewed maximum liklihood (ML) 
decoding and LP decoding in Sec.|31 in Secs.|l]and[niwe will explain why minimal codewords 
and minimal pseudo-codewords are important in the understanding of the performance of 
ML and LP decoding, respectively. Sec. El will present pseudo-weight spectra of some 
selected codes based on finite geometries. Then, in Sec. [7| we will analyze the possible 
pseudo- weights of pseudo-codewords and we will give upper and lower bounds on the pseudo- 
weight of certain vectors. In Sec. IHlwe introduce the concept of effective minimal pseudo- 
codewords and in Sec. IHlwe study the structure of minimal pseudo-codewords. Finally, in 
Sec. El we offer some conclusions. 

In the following, M, and M++ will be the set of real numbers, the set of non-negative 
real numbers, and the set of positive real numbers, respectively. Moreover, the support of a 
vector X will be defined as supp(x) = {i | 7^ O}, the Hamming weight of a vector will be 
as usual W}i{'x.) = \ supp(x)|, and (a, b) = J2i '^ibi will denote the standard inner product of 
two vectors of equal length. 



2 The Code Families under Investigation 

The codes under investigation come from the families of codes that were called type-I PG- 
LDPC and type-I EG-LDPC codes in 0. Type-I PG-LDPC codes are defined as follows. 
Let g = 2** for some positive integer s and consider a (finite) projective plane PG(2, q) (see 
e.g. [Hlini) with -\-q + l points and q^ + q + l lines: each point lies on g -|- 1 lines and each 
line contains q + 1 points.^ A standard way of associating a parity-check matrix H of a 
binary linear code to a finite geometry is to let the set of points correspond to the columns 
of H, to let the set of lines correspond to the rows of H, and finally to define the entries of 
H according to the incidence structure of the finite geometry. In this way, we can associate 
to the projective plane PG(2,g) the code CpG(2,g) with parity-check matrix H = HpQ(2,g), 
whose parameters are: 

length n = q^ + q + 1, 

dimension = n — 3* — 1, 

minimum Hamming distance dmin = ^ + 2, 
uniform column weight of H Wcoi = q+l-, 
uniform row weight of H "if row = '7+1) 

size of H n X n. 

In the usual way, we associate a Tanner graph T(HpQ(2,(j))) CHI) to parity-check matrix 
HpG(2,ij): this graph consists oi n = q'^ + q + 1 variable nodes of degree Wcoi = q + I and of 
n = q'^ + q + 1 check nodes of degree tfrow = q + 1- 

^Note that the "2" in PG(2,g) stands for the dimensionaUty of the geometry, which in the case of planes 
is 2. 



2 



Type-I EG-LDPC codes are defined as follows. Let q = 2^ for some positive integer s 
and consider a (finite) Euclidean plane EG(2,g) (see e.g. (3101) with points and + q 
lines: each point lies on q + 1 lines and each line contains q points. We essentially use the 
same procedure as outlined above in order to associate a parity-check matrix to a finite 
geometry. But before doing this, we modify the Euclidean plane slightly: we select a point 
of EG(2, q) and remove it together with the q + 'i lines through it. Doing so, we obtain 
an EG(2, q)-based code Ceg(2,ij) with parity-check matrix H = Heg(2,(j)! whose parameters 
are: 

length n = — 1, 

dimension k = n — 2>'^ + \, 

minimum Hamming distance dmin = Q + ^i 
uniform column weight of H Wcoi = q, 
uniform row weight of H Wmw = Q, 

size of H n X n. 

Again, we can associate a Tanner graph T(HEG(2,g))) ilQj) to the parity-check matrix 
HEG(2,g): this graph consists oi n = q^ — 1 variable nodes of degree iticoi = Q and of 
n = q'^ — 1 check nodes of degree Wrow = Q- 

Both families of codes have the nice property that, with an appropriate ordering of 
the columns and rows, the parity-check matrices are circulant matrices, meaning that 
CpQ(2,g) and C-^G{2,q) are cyclic codes. This fact can e.g. be used for efficient encoding. 
Such symmetries can also substantially simplify the analysis. Note that the automorphism 
groups of CpG(2,g) and C-^G{2,q) contain many more automorphisms besides the cyclic-shift- 
automorphism implied by the cyclicity of the codes. 



3 ML and LP Decoding 

In this section we briefly review ML and LP decoding ^ [2j. Consider a binary linear 
code C C {0, 1}" of length n and dimension k that is used for data communication over a 
memoryless binary-input channel with channel law PY\x{y\^)- The transmitted codeword 
will be called x = (xi, . . . , Xn) and the received vector will be called y = (j/i, . . . , Based 
on the received vector, we can define the log-likelihood ratio (LLR) vector A = (Ai, . . . , A„) G 
(R U {iboo})" to be the vector containing the LLRs Aj = log {pY\x{yi\^) /PY\x{yi\^)) ^ * = 
1, . . . , n. Using the canonical embedding of the set {0, 1} into M and of the set C into M", 
ML decoding can then be cast as 

x = argmin (x, A). (1) 

Letting conv(C) be the convex hull of C in M", the above ML decoding rule can also be 
formulated as 

x = arg min (x. A). (2) 

xGconv(C) 

Unfortunately, for most codes of interest, the description complexity of conv(C) grows ex- 
ponentially in the block length and therefore finding the minimum in @ with a linear 
programming solver is highly impractical for reasonably long codes. ^ 

^Exceptions to this observation include for example the class of convolutional codes with not too many 
states. 
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A standard approach in optimization theory and practice is to replace a minimization 
problem by a relaxed minimization problem, in our case we replace the minimization over 
conv(C) by a minimization over some easily describable polytope V which is a relaxation of 
conv(C): 

X = argmin (x, A). (3) 

If V is strictly larger than conv(C) then the decision rule in (jSJ obviously represents a sub- 
optimal decoder. A relaxation that works particularly well for LDPC codes is given by the 
following approach ^Ej. Let C be described by an m x n parity-check matrix H with rows 
hi, h2, . . . , hm- Then the polytope V = 'P(H), in this context also called the fundamental 
polytope jSl E] , is defined as 

m 

P = Pi conv(Cj) with Cj = {x G {0, 1}'^ | (h^, x) = mod 2}. 
i=i 

Note that "P is a convex set within [0, 1]" that contains conv(C), but whose description 
complexity is typically much smaller than the one of conv(C). Points in the set V will 
be called pseudo-codewords. Because the set V is usually strictly larger than conv(C), it 
can obviously happen that the decoding rule in ^ delivers a vertex of V that is not a 
codeword. Such vertices that correspond to pseudo-codewords that are not codewords are 
the reason for the sub-optimality of LP decoding (cf. P [21 El EI)- Note that V = 7'(H) is 
a function of the parity-check matrix H that describes the code C. Different parity-check 
matrices for the same code might therefore lead to different fundamental polytopes. It is 
worthwhile to remark though that all these fundamental polytopes have the property that 
'P(H)n{0, 1}" = C, i.e. all points of 'P(H) with integral coordinates are also codewords 

4 Minimal Codewords 

In this section we will discuss minimal codewords and explain their importance with respect 
to ML decoding. Although ML decoding is often impractical, knowing bounds on the block 
error rate of an ML decoder can help in assessing the performance of sub-optimal but 
practical decoding algorithms. 

Definition 1 Let C he a linear code. A non-zero codeword x & C is called minimal if and 
only if its support does not ( strictly ) contain the support of any other non-zero codeword as 
a proper subset. The set of all minimal codewords of C is commonly denoted by A4{C). □ 

We will henceforth assume that we transmit a binary linear code C over a binary-input 
output-symmetric channel (cf. e.g. Def. 1]). For this setup, when studying the ML 
decoder in or ((2)) , we can without loss of generality assume that the zero codeword was 
sent, because all decision regions are congruent. The importance of minimal codewords lies 
in the following considerations. 

Theorem 2 (cf. e.g. jl2] ) Let C be a binary linear code of length n and for x G C let 
Vf^ ^ { A G M" I (x'. A) ^ (x. A) for all x' G C \ {x}} 
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be the region in the LLR space where the ML decoder decides in favor of the codeword x.^ 
Then the decision region of a codeword x G C shares a facet^ with the decision region 
Pq'^ of the zero codeword if and only if "x. is a minimal codeword. 

Therefore, knowing the minimal codewords of the code C is sufficient in order to assess its 
ML decoding performance. In the following, we mention a few known facts about minimal 
codewords of codes that helped us in our investigations in the later chapters of this paper. 

Lemma 3 ( |14l I15p Let M.{C) denote the set of all minimal codewords of a binary [n, k, d\ 
code C. Then: 

1. M.{C) spans C. 

2. Ifli{U) denotes the submatrix o/H with columns indexed by the set U C {1, 2, . . . , n}, 
we have that a codeword Ji. £ C is minimal if and only if rank (H(supp(x))) = 
I supp(x)| — 1 = wu{x) — 1- 

3. Ifxe M{C) then \ supp(x)[ = wh(x) ^n - k + 1. 

4- If X C and |supp(x)| = wh(x) ^2d — l then x € M{C). 

5. Every non-minimal codeword can be written as a sum of two non-zero codewords with 
disjoint supports. □ 

Let us mention some further results about minimal codewords (here we also include 
non-binary codes). The problem of classifying the minimal codewords is completely solved 
only for q-aiy Hamming codes, for the second order binary Reed-Muller codes RM(2,m), 
and for the Z4 Kerdock codes ^1^1 EI? and partially for the binary Reed-Muller code of 
order r, RM(r, m) |18j . However, in general it seems to be quite difficult to describe the set 
of minimal codewords for an arbitrary linear code even in the binary case. 

We remark that in the context of linear codes, Hwang ,14^ was the first to consider the 
set of minimal codewords of a code (there called the "projecting set of a code"). He studied 
them in connection with two modifications of the correlation decoding algorithm.^ 

Minimal codewords and their properties arise also in connection with secret sharing. As 
it was first pointed out in ^7]) minimal vectors in a linear code completely specify the access 
structure of the linear secret sharing scheme defined by the code. This line of research was 
pursued in |151 116j . We finally note that minimal vectors were also studied in combinatorics 
under the concept of cycles of linear matroids. 

5 The Fundamental Cone, Minimal Pseudo-Codewords, and 
Spectra 

In this section we will shift our attention to LP decoding and the objects of interest: pseudo- 
codewords and in particular the minimal pseudo-codewords. For LP decoding of a binary 
linear code that is used for data transmission over a binary-input output-symmetric channel, 
it is sufficient to consider the part of the fundamental polytope V around the vertex 0, 

^We assume that during ML decoding ties between decoding regions are resolved randomly. 

facet is an n — 1 dimensional face of a polytope, see e.g. 
^In the light of the ML decoder as formulated in these algorithms can be seen as variations of the 
simplex method (cf. e.g. ^\) that minimizes (x, A) over the polytope conv(C). 
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cf. pi i-e. the fundamental cone. (See also that discuss this so-called "C-symmetry" 
property) . 

Lemma 4 HI Uj) Let C be an arbitrary binary linear code and let H be its parity-check 
matrix. We let J = 17(11) he the set of row indices of H and we let I = 2r(H) he the set of 
column indices o/H, respectively. For each j € J^, we let Ij = Xj(H) = |i G T | hji = l}. 
Let the fundamental cone /C(H) o/H be the conic hull of the fundamental polytope 'P(H). 
Then, /C(H) is the set of vectors u € M" that satisfy 

Vj ^ J, Vi G Tj : ^ tjj/ ^ uji, (4) 
i'ei,\{i} 

Vi G T : ^ 0. (5) 

□ 

We note that if u is in /C(H), then also a • a; is in /C(H) for any a G Moreover, 
for any a; in /C(H) there exists an a G K+-|- (in fact, a whole interval of a's) such that a ■ u 
is in P(H). 

Let = { A G M" I {u, A) ^ for all u G P(H) \ {0}} be the region where the LP 
decoder decides in favor of the codeword 0.^ It can easily been seen that = {A G 
M" I (cj. A) ^ for all cj G /C(H)}. Therefore, when studying LP decoding it is enough to 
know /C(H); all vectors u G /C(H) will henceforth be called pseudo-codewords. Moreover, 
two pseudo-codewords where one is a positive multiple of the other will be considered to be 
equivalent. 

A class of pseudo-codewords that will be used a few times later on is the class of so-called 
unsealed pseudo-codewords I^Ul |3| . These pseudo-codewords have integer entries and are 
derived from codewords in finite covers of the Tanner graph T{H). An important property 
of an unsealed pseudo-codeword u is that cj (mod 2) G C.^ 

Another important class of pseudo-codewords is the class of so-called minimal pseudo- 
codewords: 

Definition 5 (|3llll) Let C be an arbitrary binary linear code described by the parity-check 
matrix H whose fundamental cone is /C(H). A vector u G /C(H) is called a minimal pseudo- 
codeword if the set {a ■ u \ a & M+} is an edge o//C(H). The set of all minimal codewords 
will he called A4p(/C(H)).s □ 

For a given binary linear code C with parity-check matrix H, the importance of the 
set A^p(/C(H)) lies in the following fact. From basic cone properties (cf. e.g. ^21 ), it can 
easily be seen that V^^ = {A G M" | {uj,X) ^ for all a; G Mp(/C(H))}. Therefore, 
the set A^p(/C(H)) completely characterizes the behavior of the LP decoder. It can be 
shown Q that for any minimal pseudo-codeword u: there is an a G such that au: is an 
unsealed pseudo-codeword, which, among other things, implies that all components of au: 
are integers. 

Note that the above notion of minimal pseudo-codewords generalizes the notion of min- 
imal codewords in the following sense: whereas minimal pseudo-codewords correspond one- 
to-one to the edges of 'P(H) (or /C(H)) around 0, the minimal codewords correspond one- 

®We assume that during LP decoding ties between decoding regions are resolved randomly. 
''See |2UI m for an exact definition of unsealed pseudo-codewords and their properties. 
*Note that this definition implies that ^ Xp(A;(H)). 
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to-one to the edges of conv(C) around 0. (Minimal codewords are usually also minimal 
pseudo-codewords, but not always, as was remarked in JT.) 

Because of the one-to-one relationship between parity-check matrices and Tanner graphs, 
the fundamental cone can also be seen as a function of the Tanner graph representing a 
code. Therefore, in order to emphasize the dependence of minimal pseudo-codewords on 
the representation of the code, we will sometimes talk about the minimal pseudo-codewords 
of a Tanner graph. 

The fundamental cone is independent of the specific memoryless binary-input channel 
through which we are transmitting, however, the influence of a pseudo-codeword on the LP 
decoding performance depends very much on what channel is used. This influence will be 
measured by a channel-dependent pseudo-weight of pseudo-codewords; these pseudo- weights 
can be seen as generalizations of the Hamming weight that has traditionally been used to 
assess the performance of a code under ML decoding. 

Definition 6 Letu € M" . The binary-input additive white Gaussian noise channel (AWGNC) 
pseudo-weight \2^A ^ of u is defined to he 

^AWGNC(^) A Ml , 

if LJ ^ and w^^^^{u) = otherwise, where \\uj\\i and \\uj\\2 o-^e the L\- and 11.2-norm 
of u), respectively. Let u' £ M" be a vector with the same components as u but in non- 
increasing order. Introducing 

f{0 = uj'i {i-K^^i, 0<^^n), 
Jo 




the BSC pseudo-weight IM\^ is defined to be w^^^{u) = 2e if u ^ and w^^'^{u) = 
otherwise. Finally, the BEC pseudo-weight \2'J\. ^ is defined to be 

wf'^{u) = \suvv{^)\- 

□ 

Note that for x E {0, 1}" we have w^^'^^'^{yi) = tUp^^(x) = Ti)p^^(x) = wrIx). Let us 
briefly comment on the significance of the above pseudo-weights. When transmitting over 
an AWGNC, it can be shown that the squared Euclidean distance from the point +1 in 
signal space, which corresponds to the codeword 0, to the plane {A G M" | {u),X) = O} 
When transmitting over a BSC, the LP decoder decides in favor of and 
against u if the number of bit-flips on the BSC is smaller than w^^^{ijj)/2; on the other 
hand, there is at least one pattern with at least w^^^{u)/2 bit flips such that the LP decoder 
decides in favor of and against 0, assuming that ties are resolved randomly. Finally, when 
transmitting over a BEC, the LP decoder decides in favor of and against u if the number 
of erasures on the BEC is smaller than Wp^^(a;); on the other hand, there is a pattern with 
u'p^*^(a;) erasures such that the LP decoder decides in favor of u and against (assuming 
that ties are resolved randomly). For a more detailed discussion, see j4||23|. 
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Definition 7 Let C be an arbitrary binary linear code. We recall the definition of the 
codeword weight enumerator 



xeC 

and define the minimal codeword weight enumerator, and the minimal pseudo-codeword 
AWGNC pseudo- weight enumerator, to be, respectively, the polynomials (with potentially 
non-integer exponents) 

xeA4(C) 

^mpcw,AWGNC^^>j ^ ^ ^«,awgnC(^) 

MeA4p(/C(H)) 

The summation in the last enumerator is over all equivalence classes of minimal pseudo- 
codewords.^ (The minimal pseudo-codeword BSC pseudo- weight enumerator and the min- 
imal pseudo-codeword BEC pseudo-weight enumerator are defined analogously.) □ 

Instead of "weight enumerator" and "pseudo-weight enumerator" we will frequently 
use the words "weight spectrum" or "pseudo-weight spectrum", respectively, or simply 
"spectrum". Ideally, for a code defined by a parity-check matrix H, we would like to know 
all the terms of the spectra that were defined in Def. [TJ. Often, we have to settle with less, 
in particular one often focuses on some quantities that characterize important aspects of a 
spectrum. 

One such quantity is the minimum pseudo- weight: we let i(;p^*^^*~^'™"(H), it;p^^'™™(H), 
and Wp^^'™™(H) be the minimum AWGNC, BSC, and BEC pseudo-weights of a parity- 
check matrix H, i.e. the minimum of the respective pseudo-weights, over all non-zero 
points in ^(H). Using some simple tree-based techniques, or Th. 1 in [211, 
show that Wp '™™(HpQ(2,g)) ^ g + 2 and because this lower bound matches the min- 
imum Hamming weight, we actually know that ii'p^^'^^'™™(HpQ(2,g)) = g + 2. Simi- 
larly, one can show that ii'p^'~^'™™(HpQ(2,g)) = tUp^*^'™™(HpQ(-2 g)) = q -\- 2 and that 



w 



AWGNC, min/TT \ BSC,min/TT \ BEC,min/TT \ , i 

[ti-EG{2,q)) = Wp VilEG(2,g) j = Wp lilEG(2,g)j = 9+1- 



Another important quantity that characterizes pseudo-weight spectra is the pseudo- 
weight spectrum gap: 

Definition 8 Let C be an arbitrary binary linear code described by the parity-check matrix 
H and let A^p(/C(H)) be the set of all minimal pseudo-codewords that are not multiples of 
minimal codewords. We call the real-valued quantity 

AWGNC A u;^WGNC(^) _ ^gin(C(H)) 

the AWGNC pseudo-weight spectrum gap o/H. (The BSC pseudo-weight spectrum gap and 
the BEC pseudo-weight spectrum gap are defined analogously.) □ 

^Two pseudo-codewords lj,u:' £ /C(H) are in the same equivalence class if there exists an a £ M++ such 
that u> — a ■ Lj' . 
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Using [3 Cor. 8] (see also Sec. 7]), one can show that for a randomly constructed 
(ifcoh ifrow)-regular code with 3 ^ Wcoi < Wmw the AWGNC pseudo-weight spectrum gap 
becomes strictly negative with probability one as the block length goes to infinity. However, 
slightly extending the arguments that lead to the above mentioned minimum pseudo- weight 
results, one can show that for the PG(2, q)- and EG(2, g)-based codes (with square parity- 
check matrix as discussed in Sec. ^ the pseudo-weight spectrum gap is non-negative for 
finite q. In fact, we will see that for the codes investigated in SecElthe pseudo-weight spec- 
trum gap is significantly positive. We note that by applying simple performance bounding 
techniques it can be shown that the larger the gap is, the closer is the LP decoding perfor- 
mance (and potentially also the iterative decoding performance 011]) to the ML decoding 
performance as the SNR goes to infinity. ^'^ 



6 Examples of Spectra 

In this section we exemplify many of the objects that were defined in the previous chap- 
ters, i.e. we present minimal pseudo-codewords, weight enumerators, and the pseudo-weight 
spectrum gap for some short PG(2, q)- and EG(2, g)-based codes. 



6.1 Type-I PG-LDPC Code for g = 2 



The PG(2, 2)-based code CpG(2,2) has parameters [n=7, k=3, (imin=4] and can be represented 
by the following circulant parity-check matrix HpQ(2,2) of size 7x7: 



H 



PG(2,2) 



/I 


1 





1 








o\ 





1 


1 





1 














1 


1 





1 














1 


1 





1 


1 











1 


1 








1 











1 


1 


\1 





1 











1/ 



(6) 



It is not difficult to find out that the set A4(CpG(2,2)) of minimal codewords consists of the 
following codewords: 

(1,0,0,1,0,1,1), 
(1,1,0,0,1,0,1), 
(1,1,1,0,0,1,0), 
(0,1,1,1,0,0,1), 
(1,0,1,1,1,0,0), 
(0,1,0,1,1,1,0), 
(0,0,1,0,1,1,1). 

Obviously, all of them have Hamming weight 4 and they are all cyclic shifts of each other. 
Because the code has 2^ = 8 codewords in total, it turns out that this code is special in the 
following sense: there are no non-zero codewords that are not minimal codewords. 



^°Of course, the pseudo- weight spectrum gap is only a first approximation to how quickly the LP decoding 
performance approaches the ML decoding performance as the SNR goes to infinity. A better approximation 
is given by initial parts (or the whole) minimal pseudo-codeword pseudo-weight enumerator. 



9 



The set A^p(/C(HpQ(2.2))) of minimal codewords contains all the elements from A4(CpG(2,2)) 
plus the following pseudo-codewords that are not codewords (we show one representative 
per equivalence class): 

(1,2,2,1,2,1,1), 

(1.1.2.2.1.2.1) , 

(1.1.1.2.2.1.2) , 

(2.1.1.1.2.2.1) , 

(1.2.1.1.1.2.2) , 
(2,1,2,1,1,1,2), 
(2,2,1,2,1,1,1). 

All these minimal pseudo-codewords that are not codewords are cyclic shifts of each other. 
The weight enumerators are given by: 



mpcw,AWGNC(^) = 7X' + 7X^-'' , 

■"PG(2,2) ^ ' ' 

mpcw,BSC.^x 7^4 



PG(2,2) 

mpcw,BEC.^x = 7X^ + 7X\ 

Hence, the pseudo-weight spectrum gaps are n^WGNC _ Q 25 — 4 = 2.25, = 5 — 4 = 

, r- or- o 1- ^■H.PG(2.2) ' ^"■pQ^2,2) 

1, and gW^ =7-4 = 3. 

The codes introduced in Sec. |21 were based on square parity-check matrices. However, 
the code PG(2, 2) can also be described by a parity-check matrix of size 4x7 such as 



HpG(2,2) 



/I 1 1 0\ 

110 10 

110 10 

yo 1 1 1/ 



(7) 



which is the matrix formed by the first four lines of the HpQ(2,2) • Because Hp^^g 2) 
tains a subset of the rows of HpQ(2,2) it is clear that /C(HpQ(2^2)) ^ ^(-f^PG(2 2))' Moreover, 
note that a minimal pseudo-codeword of Ar(HpQ(2,2)) does not need to be minimal pseudo- 
codeword of /C(HpQ^2 2))- Iiideed, the set of minimal pseudo-codewords that are not code- 
words turns out to be the union of the following sets (in which we show one representative 
per equivalence class): 

{(3, 2, 1,1, 1,0,0)}, {(0,1, 2, 1,1,3,0)}, {(0,1, 1,1, 2, 0,3)}, {(0,1, 1,1, 1,0,0)}, 
{(2,1,1,1,0,0,1), (1,2,1,1,1,0,0), (0,1,2,1,1,1,0)}, 
{(0,1,1,1,0,2,1), (1,0,1,1,1,0,2), (2,1,0,1,1,1,0)}, 
{(0,1,1,1,2,0,1) (1,0,1,1,1,2,0), (0,1,0,1,1,1,2)}, 

where cyclic shifts of the same pseudo-codeword are grouped in the same set. It is interesting 
to see that, for HpQ^2 2)' ^ cyclic shift of a minimal pseudo-codeword is not necessarily a 
(minimal) pseudo-codeword, as it was in the case of the matrix HpG(2,2) • 



10 



It follows that 



mpcw,AWGNC(^^ = 11X4 + g^4.5^ 

PG(2,2) 

PG(2,2) 
PG(2,2) 

and that the pseudo-weight spectrum gaps are g^^^^ =4 — 4 = 0, g^'^ = 3 — 4 = 

PG(2,2) ■"■PG(2,2) 

— 1, and = 4 — 4 = 0. Comparing the enumerator •^^^'^^'^^^'^^(X) with the 

■"■PG(2,2) ■npG(2,2) 

enumerator ^™p™'^wgnc^^-j QT^^iaieni that the performance of LP decoding using the 

PG(2,2) 

second representation will be worse than the performance of LP decoding using the first 
representation. Based on iterative decoder simulations, MacKay and Davey |25| Sec. 4] 
observed a similar performance hierarchy between different representations of the same 
code. Note that the code under investigation in [23 was the PG(2, 16)-based code. Similar 
statements can be made for the BSC and the BEC. 

Before concluding this subsection, let us comment on the vector u = (0, 1, 1, 1, 1,0,0), 
which is a minimal pseudo-codeword for Hp^^g 2) ^ codeword for CpG(2,2)) even 

though it has only and 1 components. From our remarks after Lemma IH it follows that 
u cannot be an unsealed pseudo-codeword because uj (mod 2) is not a codeword. However, 
its equivalent (0, 2, 2, 2, 2, 0, 0) is an unsealed pseudo-codeword, and it stems from a triple 
cover. 

Non-codeword pseudo-codewords that contain only zero and one components will be 
discussed again in Th. El 

6.2 Type-I PG-LDPC Code for g = 4 

The parity-check matrix HpQ(2,4) of the PG(2, 4)-based code CpQ(2,4) has size 21 x 21, 
uniform column and row weight 5, and yields a code with parameters [n=21, /c=ll, dmin=6]. 
The codeword weight enumerator and the minimal codeword weight enumerator are 

^CpG(2 4)(^) = + 168X^ + 210X^ + 1008X^0 + 280X^2 ^ agox^^ + 21X^^ , 
^Cp™(2,4)(^) = 168^6 + 210X8 + lOOSX^o, 

respectively. Looking at these enumerators we see that all codewords with Hamming weight 
6, 8, and 10 are minimal codewords. Analyzing the set of all weight-6 codewords one sees 
that they all have the same pattern, i.e. they can all be obtained from a single weight-6 
codeword by applying a suitable PG(2, 4)-automorphism. The same is true for all other 
sets of codewords with the same weight. This makes the classification of all the codewords 
of CpG(2,4)) and in particular of the minimal codewords of CpG(2,4), relatively easy. 

T , J f- • ■ 4.U f If mpcw,AWGNC / TAX mpcw,BSC / i mpcw,BEC/TA\ 

instead of givmg the formula for y„ X , X , and Yxi X ), 

we simply show their histogram, cf. Fig. □ Without going into any details, it is apparent 
from Fig. ^ that the influence of minimal pseudo-codewords can vary depending on the 
channel that is used. (For related observations about varying influences of minimal pseudo- 
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Figure 1: Histograms of the AWGNC, BSC, and BEC pseudo-weight of minimal pseudo- 
codewords (PCWs) of the PG(2,4)-based code. (Note that the y-axes are logarithmic.) 
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codewords, see also the discussion in ■) The pseudo-weight gaps turn out to be 

AWGNC ^9_8_g^3_g 

^■"■PG(2,4) ' 

BSC = 7 _ g = I 

<G,.,4) = 11-6=5. 

We refer to the end of Sec. [5] for a discussion on the significance of positive pseudo-weight 
gaps. 

6.3 Type-I PG-LDPC Code for g = 8 

The parity-check matrix HpQ(2,8) of the PG(2, 8)-based code CpG(2,8) size 73 x 73, 
uniform column and row weight 9, and yields a code with parameters [n=73, A;=45, dmin=10]. 
Judging from some random search experiments in the fundamental cone /C(HpQ(2,8))! 
AWGNC pseudo-weight spectrum gap S'h^^^^ seems to be at least 6.0. 

6.4 Type-I EG-LDPC Code for g = 4 

The parity-check matrix Heq(2,4) of the EG(2, 4)-based code Ceg(2,4) has size 15 x 15, 
uniform column and row weight 4, and yields a code with parameters [n=15, fe=7, c?mm=5]. 
The codeword weight enumerator and the minimal codeword weight enumerator are 

yS,™ (X) = + + 30X^ + IbX'^ + 15X^ + 30X^ + 18X^° + X^^, 

XCeg(2 4) (^) = 18^^ + 30^^ + 15^^ + 1^^^ + 30^^ 

respectively. Looking at these enumerators we see that all codewords with Hamming weight 
5, 6, 7, 8, and 9 are minimal codewords. Analyzing the set of all weight-5 codewords one 
sees that they all have the same pattern, i.e. they can all be obtained from a single weight-5 
codeword by applying a suitable EG(2, 4)-automorphism. The same is true for all other 
sets of codewords with the same weight. 

The histograms in Fig. |21 correspond to various parity-check matrices that describe 
Ceq(2,4). Fig. [21 (top) shows the histogram for Xhp^^^^*^^*^^"'^)' Fig- HI (™iddle) shows the 
histogram for t^™p™'AWGNC / ^i^^^.^ Hpp /„ .s is a randomly selected 9x15 sub matrix (with 

■"■PG(2,4) t^'^K^^'t) 

column weights at least 2) of HpG(2,4) ; and finally Fig. |2l (bottom) shows the histogram for 



mpcw, AWGNC /j^N ^]^gj,g Ho^,„ is an 8 x 15 submatrix (with five columns having weight 

^PG(2,4) ^ ^ -1^^(2,4) V 

only one) of consecutive rows of the (circulant) matrix HpQ(2,4)- It can easily be seen that 
for the parity-check matrices under investigation those with more dependent rows lead to 
more favorable histograms. 

In Fig. 131 we show various decoding simulation results for data transmission over a 
binary-input AWGNC and lower and upper bounds: HEG(2,4)-ba'Sed sum-product algorithm 
decoding, HEQ(2^4)-based LP decoding, CEG(2,4)-based ML decoding, an upper bound on LP 
decoding based on a union of events upper bound, an upper bound on ML decoding based on 
a union of events upper bound, and a lower bound on ML decoding based on an inequality 
by de Caen as presented by Seguin It can be seen that thanks to the knowledge of 

minimal codewords and minimal pseudo-codewords we are able to obtain bounds that are 
very tight from a certain SNR value on. This is witnessed by the fast decreasing line labeled 
"LP UUB minus ML SLB" which shows the difference between the union upper bound on 
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Figure 2: Histogram of the AWGNC pseudo-weight of minimal pseudo-codewords (PCWs) 
of the EG(2, 4)-based code. (Note that the y-axes are logarithmic.) Top: For 15 x 15 
parity-check matrix Heg(2,4), 9u 
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Figure 3: Word error rate for various decoding algorithms together with some upper and 
lower bounds. (See main text for explanations.) 
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the LP decoder word error rate and the Seguin lower bound on the ML decoder word error 
rate. 

6.5 How the Results Were Obtained 

Let us briefly mention how the results for the minimal pseudo-codewords were obtained 
in Sees. 16.11 16.21 and 16.41 We used the program "Irs" [2H1 to search edges in cones. For 
the code CpG(2,4) Sec. 16.21 we additionally used the two-transitivity of the points of a 
projective plane in order to formulate a simpler edge-enumeration subproblem which can 
be solved efficiently and from which all the minimal pseudo-codewords can be derived. This 
goes as follows: it is clear that a minimal pseudo-codeword cannot only fulfill inequalities 
of type © with equality (because only u = does this), i.e. at least one inequality of 
type (jU must be fulfilled with equality. Because the automorphism group of PG(2, 4) is 
two-transitive (which means that for any two pair of points there exists an automorphism 
that maps the first pair to the second pair), we can pick any of the inequalities of type © to 
be fulfilled with equality. Solving the edge enumeration problem for this n — 1 dimensional 
cone is simpler and using the automorphism group we can derive from this the minimal 
pseudo-codewords of the cone of interest. 

6.6 Connection to Stopping Sets 

Before finishing this section that showed some pseudo- weight enumerators, let us comment 
on stopping set weight enumerators for PG-based codes that were investigated by Kashyap 
and Vardy These stopping set weight enumerators are tightly related to the BEG 

pseudo- weight enumerators because of the following reasons jHlSEl: the support of any 
pseudo-codeword is a stopping set whose size equals the BEG pseudo-weight of the pseudo- 
codeword. Moreover, for every stopping set there is at least one pseudo-codeword whose 
support equals that stopping set. However, some care must be taken when comparing a 
BEG pseudo-weight enumerator and a stopping set weight enumerator because the former 
is a sum over minimal pseudo-codewords whereas the latter is a sum over minimal stopping 
sets. (See also the discussion in Sec.|Slon effective minimal pseudo-codewords for the BEG.) 

7 Bounds on the AWGNC Pseudo- Weight 

In this section we will give some bounds on the AWGNG pseudo- weight. First we discuss 
some general bounds on the pseudo- weight of arbitrary vectors in R"; note that these 
bounds will depend only on the type of the vector. Then we discuss some bounds for certain 
selected pseudo-codewords that appear in the fundamental cone of PG-based codes. For 
easier notation, if no confusion arises, we will use the shorter Wp{uj) instead of Wp^'^^'^ (uj) . 

Definition 9 Let uj € W]j_ and let ti = t£{u) be the number of components of the vector uj 
that are equal to £, where £ € M+. Then, we call t = t{u) = {ti{u))£^R^ the type of u. 
(Note that in the following we do not assume that £ is a non-negative integer, only that it 
is a non-negative real number.) □ 

It follows from this definition that only finitely many t^'s are non-zero and that X^^^^ = 
\T\ = n for any u G M". Moreover, because |supp(a^)| = J2e>o^^^ W^Wi = S^^*^; 
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Iwlll = '^^i'^ti, we have 



AWGNC. s _ 



i>0 

A 



If u = a ■ u for some a G M.^^ then its type t = t(a;) is such that tai = te for all £. 
Lemma 10 Let uj € M" and let t] ^ be some arbitrary real number. Then 



n 



Wp{u) ^ 2 = 2 

equality if and only if u) = or r] = \\u\\2/\\<^\\i. 

Proof: If = then the statement is certainly true, so let us assume that u: ^ 0. The 
square of any real number is non-negative, therefore (r/||4j||i — ||<x;||2) ^ 0, with equality if 
and only if 77 = ||a;||2/|Iu^||i, which, after rearranging, gives r7^|ja;|[^ ^ 2r/|[a^||i||a;||2 — H^Hl- 
Finally, dividing by ?7^||a;||2 and using the definition of Wp{Lj), we obtain the desired result. 

□ 

Corollary 11 Let u € M", let t = t(a;) be the type of u, and let r] ^ be some arbitrary 
real number. Then 

wp{uj)^^(3ete with = ^^^1^ = i - (^i - . 

Proof: The result follows immediately from Lemma [TUl □ 

Corollary 12 Let u € M" and let t = t(ijS). Moreover, let r be the ratio of the largest 
positive I such that ti is non-zero and the smallest positive I such that t£ is non-zero. Then 
we have the lower bound 

4r 

^P^"^) ^ (^ + 1)2 • |supp(u;)|. 

This bound was also obtained by Wauer \3IJ^ using a different derivation. 

Proof: Let m be the largest positive I such that ti is non-zero and let m! be the smallest 
positive £ such that t^ is non-zero. These definitions obviously yield r = m/m! . Consider 
Cor.HUwith 7? = rn±pL. We obtain Wp{u) ^ "^.fj.tt (a) with = 4£g^ = 1 - (1 - 

Wc obscrvc that Pm' = Pm = (n^+^/)ii = (^+1)^ ■ Siucc Pi IS strictly concave in i 
we must have /?£ > Pm' = Pm = (^r+iy^ fo^ all m' < £ < m. It follows that 

4r 4r ■^-^ 4r 

VJ 



p{u)= Pete^ (r + i)2 ^'= + Yl = u + i)2 ' |s^PP(^)l- 

□ 



Under the same assumptions as in Cor. 1121 Kelley and Sridhara jSlJ proved that Wp{u:) ^ 
(i+r^Xr-i)+2r I ^^PP(^)I- Note that for r = 1 and r = 2 the bound in Cor. [T^ equals this 
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bound and that for integers r larger than 2 the bound in Cor. is larger than this bound. 
(Note that it follows from some simple considerations that for a minimal pseudo-codeword 
cj the ratio r is 1 or at least 2.) 

In the following subsections we will give bounds on the AWGNC pseudo-weight of the 
class of minimal pseudo-codewords having the property that their components are all small 
non-negative integers. Based on the examples in Sec. El we formed the belief that for the 
Tanner graph families under study that the non-codeword minimal pseudo-codewords with 
small AWGNC pseudo- weight are from this class. 



7.1 Pseudo-Codewords with Zeros and Ones 

We start our analysis with pseudo-codewords of smallest possible entries, i.e. pseudo- 
codewords with zeros and ones. The following theorem gives a lower bound on their weight. 

Theorem 13 Let H = HpQ(2,q) ^.''^d let u € /C(H) be a (not necessarily minimal) pseudo- 
codeword of type t with to non-negative, ti positive, and ti = otherwise. If uj is not a 
codeword in CpG(2,g) then 



^ + 1 + -VV + 1% + 16 



Proof: See Sec. EH □ 
Note that the above bound yields, for g = 2 and (7 = 4, Wp{u>) ^ q + 4, and, for q ^ 8, 
Wp{u) ^ q + 5, respectively. 

Remark 14 An example of a non-codeword pseudo-codeword with only zeros and ones as 
discussed in Th. 1131 was presented at the end of Sec. 16.11 Note, however, that this example 
was for HpQ^2 2) HpQ(2,2) in ©• □ 

Observations for small PG(2, g)-based codes suggest the following conjecture. 

Conjecture 15 Let H = HpQ(2,g) and let u € /C(H) be a minimal pseudo-codeword of 
type t with both to non-negative, ti positive, and tg = otherwise. The vector u is then a 
minimal codeword. 

The following theorem generalizes part 3 of Lemma El (regarding properties of minimal 
codewords) to pseudo-codewords with components equal only to zero and one. 

Lemma 16 Let C be an [n, k] binary linear code represented by a parity-check matrix H. 
(Note that we do not assume that C = CpG(2,ij) or C = C^G{2,q)-) Let u G A^(H) be a minimal 
pseudo- codeword of type t with to non-negative, ti positive, and ti = otherwise. Then 

Wp^Lj) = wu{<^) ^ n — k + 1. 

Proof: Without loss of generality we can assume that the pseudo-codeword indices have 
been reordered such that the first ni = W}i{u>) components of u are equal to one and such 
that the remaining n2 — n — wui^) components of are equal to zero. 

Let Kio; ^ be the collection of inequalities of type Q that a pseudo-codeword must 
fulfill and let K2^ ^ be the collection of inequalities of type © that a pseudo-codeword 
must fulfill. Then there exists a full-rank (ni — 1) x n-submatrix Ai = (An | A12) of 
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Ki and a full-rank n2 x n-submatrix of A2 — (A21 | A22) of K2 such that Aio; = and 
A2U; = 0, such that A21 = and A22 = In2) such that rankiR(A) = n — 1, where 
A = ( ^2 ) ~ ( A21 A22 ) ~ ( 0^ ) ■ "'^PP^y™? elementary row operations to the matrix 
A we obtain the matrix A = ( "^q^ jjj*^ ) . Because all non-zero entries of cj are equal to one 
and because u is assumed to be a minimal pseudo-codeword, it turns out that all rows of 
All must contain exactly two non-zero elements, one equal to +1 and one equal to —1. 

These facts can be used as follows. First, we will show that rankip'2(Aii) = ni — 1. 
Secondly, we will show that rankip'2(Aii) ^ n — k. Finally, combining these results we will 
obtain the desired statement that Wp{cj) = W}i{<^) = ni ^ n — k + 1. 

So, let us show that rankF2(Aii) = ni — 1, i.e. that rankF2(Aii) = rankiR(Aii). Indeed, 
using the the special row structure of An, it can be verified that the only vector in the 
(right-hand side) kernel of An (mod 2) is the all-ones vector over F2 of length ni. 

Secondly, let us show that rankir2(Aii) ^ n — k. Indeed, we observe that every row in 
Ai (mod 2) corresponds to a row in H. This implies that rankiF2(Aii) ^ rankiP2(Ai) ^ 
rankF2(H) ^ n — k. □ 

Remark 17 A crucial element in the above proof was the fact that all rows of An contain 
exactly two non-zero entries, one equal to +1 and one equal to —1. For minimal pseudo- 
codewords where not all non-zero entries are equal, this is not the case anymore and therefore 
we cannot use the above proof to generalize the lemma statement to other types of minimal 
pseudo-codewords. □ 

7.2 Pseudo-Codewords with Zeros, Ones, and Twos 

Theorem 18 Let H = ilpG{2,q) ^^^^ G ^(H) be of type t with both to non-negative, 

ti positive, t2 positive, and = otherwise. Then 

32 

wp{u) ^ -(g + 2)«1.185((? + 2). 

(a) (b) 

Proof: For any i G I we must have T.i'&i\{i]^i' = 12j&j,T.i'&i^\{i]^i' > T.jej,^i = 
{q + 1)0^4, where at step (a) we used the fact that all variable nodes are at graph distance 
two from each other in the Tanner graph associated to H, and where at step (b) we used 
the inequalities in (jlj). Adding uoi to both sides we obtain X^j/gjWj' ^ {q + 2)a;j. Now, fix 
an z S I for which oji = 2 holds and express X^j/gjWj' in terms of t: it must hold that 
ti + 2t2 ^ 2{q + 2) (c). 

In a second step, we construct a vector u' = (w'^, . . . , u;'^) € M" such that 

f if = 

2 \iuji = l (for ah zG J). 
1 if u)i = 2 



/ A 
OJi = 



It can easily be seen that u' lies also in the fundamental cone, i.e. u' € /C(H), and that uj' 
has type t' with t'^ = t2 positive, t'2 = ti positive, and = otherwise. In other words, 
switching 1— > 0, 1 1— > 2, 2 1— > 1 we obtain another pseudo-codeword. Arguing as above, 
for any i ^ Z we must have Yli'e.l^'i' ^ + Now, fix an i € X for which = 2 

holds, and express in terms of t': it must hold that t'l + 2^2 ^ 2(g + 2), i.e. that 

t2 + 2ti ^ 2{q + 2) (d). 
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Combining (c) and (d) we obtain 3(ti +t2) ^ 4{q+2), i.e. | supp(a^)| = ii +^2 ^ 5(9 + 2). 

^ . _L 9^ _ 8 . 4^^ _L 9^ _ 32 (, 



Using Cor. m we can conclude that Wp{u) ^ • |(g + 2) = | • |(g + 2) = p(g + 2) 



Using some stronger assumptions on the pseudo-codeword u: we can obtain a stronger 
lower bound, as is shown in the next theorem. 

Theorem 19 Let H = ilpG{2,q) C'l^-d let u € /C(H) be of type t with both to non-negative, 
ti ^ q + 2, t2 positive, and tg = otherwise}^ Then 

Wp{u) ^ + « 1.333((7 + 2). 

Proof: The start is similar to the beginning of the proof of Th. 1181 For any i € T we must 
have ^j/gjCJj' ^ (g + 2)uji. Now, fix an i e X for which oJi = 2 holds, and express Yli'&i^i' 
in terms of t: it must hold that ti + 2t2 ^ 2{q + 2), or, equivalently, t2 q + 2 — ti/2. For 
any 77 7^ we obtain 

(a) (2r? - l)ti + (4r/ - 4)t2 (b) (2r? - + (4r/ - 4)((7 + 2 - ti/2) 
Wp{u) ^ 2 ^ 2 

^ ti + (477-4)(g + 2) 

where at step (a) we used Cor. ^2 and at step (b) we used the inequality on t2 that we 
just found above. Using the assumption that ti q + 2 from the theorem statement we 
get Wp{u) ^ The right-hand side of this expression is maximized by r/* = |: 

inserting this value yields the lower bound in the theorem statement. 

□ 

Remark 20 Let C be the code defined by H. If a pseudo-codeword is an unsealed pseudo- 
codeword [20115 then it is equal (modulo 2) to a codeword of C. Therefore, the number of 
odd components of an unsealed pseudo-codeword must either be zero or at least equal to 
the minimum Hamming weight of the code. So, if we actually know that u in Th. is 
an unsealed pseudo-codeword then the requirement ti ^ g + 2 in the theorem statement is 
equivalent to the requirement ti ^ 1. 
Note also that Th. [H 

can be generalized to the setup where u G A^(H) has type t with 
to non-negative, tm positive for some integer m ^ 2, ti non-negative for 1 ^ i ^ m — 1, 
te = Ofoi£^m + l, and Eodde^i >q + 2. Then Wp{u) ^ ^^^^^ iq + 2). □ 



Example 21 Let us exhibit some low- weight minimal pseudo-codewords that contain only 
zeros, ones, and twos. Consider first the case q = 2. The projective plane for g = 2 is shown 
in Fig. 0](a): it has 7 points and 7 lines and we consider the points to be variables and the 
lines to be checks. Fig. ^ (a and b) shows two codewords of weight g -|- 2 = 4; note that 
their supports overlap in ^±2 = 2 positions. Adding these two codewords together yields 
the pseudo-codeword shown in Fig. 21(c). Switching the zero value into a two results in the 
pseudo-codeword in Fig. |ll(d); it can be checked that this pseudo-codeword is actually a 

^^See Rem. 1^ for a comment on these conditions. 
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Figure 4: Codewords and pseudo-codewords used in Ex. |^ 

minimal pseudo-codeword. It has AWGNC pseudo-weight 6.25, whereas the lower bounds 
in Ths. [181 and [191 are 4.74 and 5.33, respectively. 

Similarly, in the case of g = 4 it is possible to start with two codewords of weight 
(7 + 2 = 6 whose supports overlap in = 3 positions. After adding them and switching 
two zeros (that are specifically chosen and lie on the same line) into two twos, one gets 
a minimal pseudo-codeword of AWGNC pseudo- weight 9.85, whereas the lower bounds in 
Ths. El and El are 7.11 and 8.00, respectively. 

In the case g = 8 it is possible to start with two codewords of weight q + 2 = 10 
whose supports overlap in = 5 positions. After adding them and switching three zeros 
(that are specifically chosen and form a triangle) into two twos, one gets a minimal pseudo- 
codeword of AWGNC pseudo-weight 16.10, whereas the lower bound in Ths. 1181 and 1191 are 
11.85 and 13.33, respectively. 

We conjecture that, with suitable generalizations, the above construction can be ex- 
tended to larger q. □ 

7.3 Pseudo-Codewords with Zeros, Ones, Twos, and Threes 

Theorem 22 Let H = ilpG{2,q) C'l^d let u € /C(H) be of type t with both to non-negative, 
ti positive, t2 non-negative, t^ positive, and t^ = otherwise. We require that u: is an 
unsealed pseudo-codeword. Then 

Wp{u)^l-{q + 2) = 1.125iq + 2). 

o 

Proof: The start is similar to the beginning of the proof of Th. 1181 For any z G X we must 
have Yli'eJ^i' ^ (? + 2)u;i. Now, fix an i E T for which = 3 holds and express ^j/gjWj' 
in terms of t: it must hold that ti + 2t2 + 3t3 ^ 3{q + 2) (a). 

In a second step, we construct a vector u' = {iu[, . . . , u;^) € M" such that 

3 if Ljj = 1 

(tor all t £ 1). 

1 if LOi = 3 
otherwise 

It can be seen that cj' lies also in the fundamental cone, i.e. u' £ /C(H),^^ and that lj' 
has type t' with t'l = t^ positive, t'2 = ^2 non-negative, t'^ = ti positive, and = t£ 

^■^Note that the inequality 3 + 3^1 goes into the inequaUty 1 + 1^3, which is wrong. However, we 
assumed that w is an unsealed pseudo-codeword, which, among other things, implies that the modulo-2 sum 



/ A 

OJ4 = 



21 



otherwise. Arguing as above, for any i G X we must have J2i'ex^i' ^ + Now, fix 

an z G T for which lo'- = 3 holds and express ^j/gjWj' in terms of t': it must hold that 
t[ + 2t'2 + 3t'^ ^ 3(g + 2), i.e. that + 2^2 + 3ti ^ 3{q + 2) (b). 

Combining (a) and (b) we obtain 4:{ti+t2 + ts) ^ 6(g + 2), i.e. |supp(a;)| = ti + t2 + i3 ^ 
|(g+2). Using Cor. 1121 we can conclude that w;p(^) ^ j0^-^{q+2) = | -1(9+2) = |(g+2). 

□ 

8 Effective Minimal Pseudo-Codewords 

The BSC can be seen as a binary-input AWGNC where the values at the output are quan- 
tized to +1 or —1. It follows that the components of the log-likelihood vector A can take 
on only two values, namely +L and — L, where L is a positive constant that depends on the 
bit flipping probability of the BSC. Because of this quantization, there are certain things 
that happen for the BSC that cannot happen for the AWGNC. Similarly, there are things 
that happen for the BEC that cannot happen for the AWGNC or the BSC. By introducing 
the so-called effectiveness of minimal pseudo-codewords, this section discusses some of these 
special behaviors. 

Definition 23 Fix a memoryless binary-input channel and let C^'^^ C (M U {±00})" be the 
set of all possible log-likelihood ratio vectors upon sending the all-zero codeword}^ Moreover, 
let us fix a parity- check matrixH and Zet A4p(/C(H)) be the set of minimal pseudo-codewords. 
A minimal pseudo-codeword u € 7Wp(/C(H)) is called effective of the first kind for that 
particular channel if there exists a A € such that < and {uj',X) ^ for all 

uj' £ Alp(/C(H)) \ {i^}. A minimal pseudo-codeword u G A^p(/C(H)) is called effective of 
the second kind for that particular channel if there exists a A € such that {uJ,X) ^ 
and {u', A) ^ for all u' € A^p(/C(H)) \ {uj}. ( Obviously, a minimal pseudo-codeword that 
is effective of the first kind is also effective of the second kind.) □ 

Let £q"^ C be the set where LP decoding decides in favor of the codeword 0. 
From the above definition it follows that a minimal pseudo-codeword "shapes" the set 

(n) 

£q if and only if it is an effective minimal pseudo-codeword. More precisely, in the case 
where a minimal pseudo-codeword cj is effective of the first kind then there exists at least 
one A G jC'^"-' where a; wins against all other minimal pseudo-codewords (and the zero 
codeword). Moreover, in the case where a? is effective of the second kind we are guaranteed 
that there is at least one A € were u: is involved in a tie; if and how often u wins 
against all other minimal pseudo-codewords (and the zero codeword) depends on how ties 
are resolved. 

of the tJi's that are involved in a check is zero. Therefore, it cannot happen that the non-zero tj^'s that are 
involved in a check have the values 3, 3, and 1. 

"For the AWGNC we have C^^^^^c = for the BSC we have d^^^ = {±L}" for some L € R++, 
and for the BEC we have -CgEC ~ {'^' +00}" • Please note that there was a slight mistake in Def. 10], 
i.e. in 1521 Def. 10] we forgot to require that the all-zeros codeword was sent. Nevertheless, compared to \'6'2\ 
Def. 10], the sets ^awcnc ^'^'^ '^bsc remain unchanged, whereas for the BEC we have {0, -l-oo}" instead of 
{—00,0,-1-00}". Moreover, we assume that L G K+-i- and not L G R+ in order to avoid the uninteresting 
case L — stemming from a BSC with cross-over probability 1/2. 

^^The fact that a minimal pseudo-codeword is effective of the second kind does of course not exclude the 
possibility that there are also A G were u: wins (unconditionally) against all other minimal pseudo- 
codewords (and the zero codeword). 
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Theorem 24 For the binary-input AWGNC and any parity-check matrix H all minimal 
pseudo-codewords o//C(H) are effective of the first kind. 

Proof: This follows from basic cone properties (cf. e.g. |21j). □ 

We now turn to the BSC. As the following observations show, for this channel not all 
minimal pseudo-codewords need to be effective of the first or of the second kind. 

Theorem 25 Consider data transmission over a BSC using the code defined by H = 
HpG(2,g) • Then LP decoding can correct any pattern of | bit flips and no pattern of more 
than q bit flips. 

Proof: Because u;p^*^'™"^(H) = q -\- 2, the BSC pseudo-weight of any pseudo-codeword in 
A^(H) is at least q + 2. Therefore LP decoding can correct at least [^-^-|— = | bit flips. 

Let us now show that LP decoding can correct at most q bit flips. Remember that a 
necessary condition for LP decoding to decode a received log-likelihood vector A to the zero 
codeword is that A) ^ for all cj G /C(H).^^ Assume that we are transmitting the zero 
codeword and that e bit flips happened. Hence e components of A are equal to —L and 
n — e components of A are equal -\-L. It can easily be checked that the following u is in 
/C(H): let = 1 if Aj = —L and cjj = 1/q otherwise. For this the condition (cj, A) ^ 
translates into e • {—L) + [n — e) ■ (l/q) ■ {+L) ^ 0, i.e. e ^ = ,^1^"^ = Q + Because 
e must be an integer this inequality turns into the inequality e ^ -|- = q. □ 

Observe that the way we constructed the pseudo-codeword u in the proof of Th. 1251 can 
be seen as a generalization of the so-called canonical completion jSJ |4| , however instead of 
assigning values according to the graph distance with respect to a single node, we assign 
values according to the graph distance with respect to the set of nodes where Aj is negative. 
(Note that the Tanner graph of H = HpQ(2,(j) has a special property: all variable nodes are 
at graph distance 2 from each other.) Such a generalization of the canonical completion 
was also used by Haley and Grant [Ml for the analysis of their codes. 

Corollary 26 Consider the code defined by Jl = HpQ(2^q). For the BSC, a necessary 
condition for a minimal pseudo-codeword u of /C(H) to be effective of the second kind is 
that q + 2s^ Wp^'^i^) ^2q-^2. 

Proof See Sec. □ 

For q = 4 it turns out that A^(HpQ(2,4)) has minimal pseudo-codewords with BSC 
pseudo-weight equal to 12. (These minimal pseudo-codewords have type t with t2 = 1, 
ti = 12, to = 8, and ti = otherwise.) Cor. clearly shows that these cannot be effective 
of the second kind for the BSC, since, for g = 4, any effective minimal pseudo-codeword of 
the second kind must fulfill 6 ^ Wp^^{uj) ^ 10. 

Judging from Fig. ^ it also seems — as far as AWGNC and BSC pseudo-weight are 
comparable — that soft information is quite helpful for the LP decoder when decoding the 
code CpG(2,4) defined by HpG(2,4)- 

We now turn to the BEC. Because A) ^ for all minimal pseudo-codewords u, no 
minimal pseudo-codeword can be effective of the first kind. 

^^Note that this is usually not a sufficient condition for correct decoding, e.g. in the case where ties are 
resolved randomly. 
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Figure 5: Part of PG(2,4) discussed in Ex.Ei 

Theorem 27 Consider data transmission over a BEC using the code defined by H. Let 
LD he a minimal pseudo-codeword such that there is an unsealed pseudo-codeword associated 
to uj with at least one odd component. Then there exists a non-zero codeword c such that 
whenever {u, A) = for some A G ^bec ^^^^ '^^■^^ (^) — 0- 

Proof: Let u' be an unsealed pseudo-codeword that is a positive muhiple of cj. By assump- 
tion, we can assume that at least one component of u:' is an odd integer. It follows 4^ that 
c = Ijj' (mod 2) is a non-zero codeword. Let A € C^^q be such that {uj,X) = 0. Because 
uj' is a positive multiple of u we must have (cj', A) = and because supp(c) C supp(a^') we 
must have (c, A) = 0. □ 

Corollary 28 Consider data transmission over a BEC using the code defined by H. Under 
block-wise ML decoding we define a block error to be the event that there is a tie among at 
least two codewords. Similarly, under LP decoding we define a block error to be the event 
that there is a tie among at least two pseudo-codewords. If for all minimal pseudo-codewords 
there exists an associated pseudo- codeword with at least one odd component then the block 
error rate of block-wise ML decoding coincides with the block error rate of LP decoding. □ 

By listing all the minimal pseudo-codewords, it can be shown numerically that the 
condition in Cor. [2H1 is fulfilled for H = HpQ(2,g) when q = 2 and q = 4. It follows that 
for these two codes block-wise ML and LP decoding yield the same block error rate (under 
the above definition of block error rate). This corroborates the observations made in |33[ 
Fig. 1] for q = A. 

9 The Structure of Minimal Pseudo-Codewords 

In this section we discuss the geometry of minimal pseudo-codewords. Minimum- weight 
codewords correspond to point-line configurations in the projective plane that have been 
studied by several authors. Let us introduce some notation and results from finite geome- 
tries, cf. e.g. fOj. A fe-arc in PC(2,g) is a set of k points no three of which are collinear. A 
k-arc is complete if it is not contained in a (/c-|- l)-arc. The maximum number of points that 
a fc-arc can have is denoted by m(2, q), and a fc-arc with this number of points is called an 
oval (in the case where q is even this is sometimes also called a hyper-oval). One can show 
that m(2, q) = q + 2 for q even and m(2, q) = q -\- 1 iov q odd. One can make the following 
two interesting observations for the case q even. Firstly, if two ovals have more than half 
their points in common, then these two ovals coincide. Secondly, if a g-arc is contained in 
an oval then the number of such ovals is one if g > 2 and two if q = 2. 

It turns out that in the case q even, the codewords with minimal weight are q -\- 2-arcs 
and therefore ovals. However, whereas the classification of ovals for odd q is simple (they 
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all correspond to conies), the ovals for even q are not classified that easily. For even q, 
one says that an oval is regular if it comprises the points of a conic and its nucleus; one 
can show that for g = 2*, irregular ovals exist if and only if s ^ 4. It turns out that the 
classification for irregular ovals is highly non-trivial.^^ So, given that even the classification 
of the codewords of minimal weight is difficult, it is probably hopeless to obtain a complete 
classification of the minimal codewords and minimal pseudo-codewords of codes defined by 
HpQ(2,g), however it is an interesting goal to try to understand as much as possible about 
the structure of these codewords and pseudo-codewords. 

In some recent papers, the structure of codewords of projective-plane-based codes has 
been discussed by Kashyap and Vardy [22] (that paper talks also about stopping sets in 
Tanner graphs derived from projective planes), by Justesen et al. [311 ESI by Laendner 
and Milenkovic [5^] (that paper also talks about trapping sets in Tanner graphs derived 
from projective planes). Moreover, the minimal- weight codewords of Euclidean-plane-based 
codes were discussed by H0holdt et al. [37]; also here these configurations are tightly related 
to ovals. However, because not all ovals are regular, the classification is not that simple also 
for these codes. 

From now on, we will only consider projective planes PG(2, q) and q will always be 
even, i.e. a power of two. Before we state our conjecture about the structure of minimal 
pseudo-codewords, let us first look at an example. 

Example 29 Let g = 4. We can find a minimal pseudo-codeword u whose type t is to = 8, 
ti = 8, t2 = 5, and ti = otherwise. This pseudo-codeword can be obtained using a 
procedure similar to the one used in Ex. 1211 Firstly, on has to add two vectors x^^^ and 
x^^) of weight 6 whose supports overlap in two positions. This yields a pseudo-codeword u 
of type t with Iq = 11, ti = 8, t2 = 2, and te = otherwise. Secondly, in order to obtain a 
minimal pseudo-codeword, one has to switch three zeros (that were appropriately chosen) 
into three twos. 

Let us analyze this procedure. Since a minimal pseudo-codeword corresponds to an 
edge of the fundamental cone, it is clear that the inequalities in ([IJ and ([SJ that are fulfilled 
with equality must form a system of linear equations of rank 21 — 1 = 20. We start with 
two minimal codewords x^^^ and x^^) that each yield a system of linear equations of rank 
21 — 1 = 20. These two codewords have been chosen such that their sum u yields a system 
of linear equations of rank 21 — 2 = 19. 

To find the three zeros that we have to switch, we proceed as follows. It turns out that in 
the projective plane PG(2, 4) there are two lines, say Li and L2, such that all the entries of 
u) that correspond to the points on these two lines are zero. Let Pq be the intersection point 
of these two lines, cf. Fig. [S] There exists a point Pi on Li and a point P2 on L2 such that 
modifying cj by assigning them the same value a ^ yields a vector in the fundamental 
cone, as long as a is not too large. In fact, for a > 2 the vector is outside the fundamental 
cone, and for a = 2 it yields a vector that is a pseudo-codeword and that yields a system 
of equations of rank 21 — 1 = 20, i.e. it is a minimal pseudo-codeword. □ 

Conjecture 30 For the Tanner graph defined by HpQ(2,g) every minimal pseudo-codeword 
is a sum of a few minimal pseudo-codewords with a change of a few low-value components 

According to |5| Th. 8.35], there are precisely two projectively distinct ovals in PG(2, 16), the so-called 
regular oval V{T^) and the so-called oval Co = T^{Fo)- Moreover, according to ^ Th. 8.36], there are 
precisely six projectively distinct ovals in PG(2,32). 

^^Note that the remark "It is also known that an oval in EG(2, g) consists of a conic and a nucleus" (which 
would imply that they are regular) in [371 Sec. 2] is wrong in general (38| . 
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such that they become the large components in the equations associated to the lines that pass 
through them. 

Hence, to find minimal pseudo-codewords, we have to take sums of two minimal pseudo- 
codewords that give rank n — 2 (if possible; lower otherwise) and change few components 
that are "not significant" into a "significant" ones. We call a component significant if it is 
the sum of the other components that belong to a line passing through the point, for most of 
such lines. 

The following conjecture is based on our observations (for small q) that among the 
minimal pseudo-codewords the ones with only zeros, ones, and twos yield non-codeword 
minimal pseudo-codewords of smallest weight. A positive answer to this conjecture would 
result in a much better understanding of the minimal pseudo-codewords in general and of 
the AWGNC pseudo-weight spectrum gap, in particular. 

Conjecture 31 Let H = HpQ(2,q) o,iT-d consider the pseudo-codewords that have minimal 

AWGNC pseudo-weight among all minimal pseudo-codewords that are not multiples of min- 
imal codewords. We conjecture that the type t of these pseudo-codewords is such that to is 
non-negative, ti is positive, t2 is positive, and ti = otherwise. (If this conjecture is not 
true, find the the smallest £ such that these pseudo-codewords have type t with ti ^ for 
i G {0,1, . . . ,i} and t^ = otherwise.) 

10 Conclusions 

We have investigated the minimal pseudo-codewords of some codes whose Tanner graphs 
are derived from projective and Euclidean planes and we have introduced the notion of a 
pseudo-weight spectrum gap for a parity-check matrix, a concept which is certainly worth- 
while to be further explored. Although our numerical results are for codes of modest length, 
to the best of our knowledge this is the first study that tries to analytically quantify the 
behavior of PG{2,q)- and EG(2, g)-based binary linear codes under LP and iterative de- 
coding. Extending these results to somewhat longer codes has the potential to explain 
many experimental observations made in the past. Moreover, we have obtained a clearer 
picture about the structure of the minimal pseudo-codewords of the Tanner graphs under 
investigation, nevertheless more work is required to get a sufficiently tight characterization 
of them. 

We also introduced the notion of the effectiveness of a minimal pseudo-codeword and 
we saw that in the case of non- AWGNC channels there are minimal pseudo-codewords that 
are not effective (of the first kind or of the second kind). Interestingly, in that context 
we were able to use the canonical completion, a tool that so far has been very useful for 
characterizing families of (u)coi) w^row) -regular LDPC codes, with Wcoi, w^row bounded when 
the block length goes to infinity, i.e. code families where the Tanner graph diameter grows 
with the block length. (This is in contrast to the PG(2, g')-based Tanner graphs which have 
girth 6 and diameter 3, independently of q). 

A Proofs 

We include here two of the larger proofs of this paper. 
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Figure 6: Part of PG(2,g) showing the relevant part of a codeword with Hamming weight 
2q. (Here for q = 4.) 




Figure 7: Part of PG(2, q) showing the points P' and P" and the hnes L' and L" that were 
used in proof of Th. 1131 (Here for q = 4.) 

A.l Proof of Theorem [H 

Let s"*" = H • cj""" (in M) and let JTodd — {i € ^7 | sj is odd} be the set of all rows of H where 
the corresponding entry in s is an odd integer. If u were a codeword then all entries of s 
would be non-negative even integers. However, because u is assumed to be a non-codeword, 
l^oddl ^ 1- Note moreover that because supp(a;) is a stopping set, we must have Sj ^ 1 for 
aU j e J. 

The proof proceeds as follows. First we will show that for any non-codeword u we must 
have I Joddl ^ 'Z + 1- Secondly, we will show that this lower bound on | J'odd! implies the 
lower bound mentioned in the theorem. 

So, let us show that |j7odd| ^ 9+1 for any non-codeword u>. The proof is by contradiction, 
i.e. if jTodd has |i7odd| ^ we will show that there exists a j £ Jodd such that the value 
of Sj must be an even integer. To that end, it turns out to be useful to reverse the usual 
interpretation of the columns and rows of H: the columns will correspond to lines and the 
rows will correspond to points of PG(2,q'). With this, there is a one-to-one relationship 
between the points of PG(2, q) and the entries of s. 

Before we proceed, consider Fig. |H1 that shows two lines in PG(2,g) and the points on 
them (here for g = 4). Letting x € be a vector where the entries corresponding to black 
dots equal 1 and the other entries equal 0, one can easily verify that x • H = (mod 2), 
i.e. X is a codeword of the code with parity-check matrix H"*". 

Now, choose any j € Joddi consider Fig. [71 and let P' correspond to the j-th row of H. 
Because |j7odd| ^ it is possible to choose a line L' through P' such that all points on it 
(except for P') have an even sj. On this line it is then possible to choose (again because 
l^oddl ^ 9) a P" 7^ P' such that there is a line L" through it such that all points on it 
have an even sj. Let x € be a vector where the non-zero entries corresponding to the 
points on L' and L" (except for P") equal 1 and the other entries equal 0. Because of the 
considerations in the paragraph above, it is clear that x • H = (mod 2), i.e. all entries in 
X • H are even integers. This implies that x ■ s""" = x ■ (H • u'^) = (x • H) • u;"'" must be an 



27 



even integer. This is a contradiction, because of the way we have chosen P', L' , P" , and 
L", the inner product x • s""" must be an odd integer. 

Let us now prove that the lower bound |t7odd| ^ q + 1 on jjToddl implies the lower 
bound on Wp{uj) mentioned in the theorem. Because of the special properties of H we have 
H"""!! = ql + J, where I is the identity matrix of size nxn and where J is the all-ones matrix 
of size nxn. Then |[s||2 = ss"*" = a;H"'"Ha;"'" = qulcj'^ + uJu)'^ = q\\uj\\2 + = qti + tf. 
On the other hand, 

We would like to find a lower bound on ||s||2. If j G J'\ jT'odd then we use X^j/gj. uJi' ^ 2uJi, 
for all i in Ij (which is implied by @), otherwise we use X^j/gj, t^i' ^ 2uji + 1 (which also 
follows from (^J, together with the observation that Sj = Yli'eij ^i' ^ •^)- 

jejieij i'eij jejieij j&Joddi&^j i€i jeJi jeJodd«Gij 

= 2{q + + E^*^2(g+l)||c^||^ + 3|XddI = 2((7 + l)ti+3|Jodd|. 

Combining the above results we obtain qti+tl = ||s||2 ^ 2(g+l)ti+3|^odd|) or, equivalently, 
tj-{q + 2)ti - 3| J'oddl ^ 0. It follows that ^ | + 1 + ^y/{q + 2y + 12|Jodd|-^^ Inserting 
the lower bound |v7oddl ^ g + 1, we obtain 

*i ^ I + 1 + ^V'z' + i^ + ie. 

Because Wp{uj) = ti and because ti is an integer, the final result follows. 
We conclude with two remarks: 

• More sophisticated considerations might lead to better lower bounds on Wp{u:); how- 
ever, note that | iToddl ^ + 1 is the best lower bound that can be given on the size 
of cTodd without additional information about the set. Namely, if jTodd happens to be 
equal to J7i for some i € T then | J'oddl = q + I and no row in H corresponding to 
an entry in J^dd can be expressed as a linear combination of rows corresponding to 
entries in J' \ J'odd- 

• The bounding techniques used in the second part of the proof were inspired by the 
bounding techniques that were used in [2^, which in turn were generalizations of |39j . 

A. 2 Proof of Corollary EEl 

Let a; be a minimal pseudo-codeword with Wp^''-^{u) > 2q + 2. The proof is by contradiction, 
i.e. we will assume that cj is effective of the second kind and then we will show that for 
any A € C^^^ with (cj. A) ^ there exists a minimal pseudo-codeword u" ^ uj such that 
< 0. 

Assume that we are transmitting the zero codeword and that bit flips happened at 
positions £. Hence l^l components of A are equal to —L and n — \£\ components of A are 

^^ti ^ 1 + 1 — + 2)^ + 12|^7odd| is not possible because we know that ti ^ q + 2. 
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equal +L. Assume that (a;, A) ^ 0. From the definition of the BSC pseudo-weight it follows 
that \£\> q + 1, i.e. \£\'^ q + 2. 

Choose a sub-pattern £' <Z £ oi bit-flips with |f '| = (7 + 1 and define the corresponding 
A'. From the definition of the BSC pseudo-weight it follows that (w, A') > (otherwise 
u>BSC(^) ^ 2g + 2). 

Similarly to the proof of Theorem 1251 we can construct a pseudo-codeword u)' based 
on £' such that u'^ = 1 \i i ^ £' and u}[ = 1/q otherwise. This pseudo-codeword has the 
property that {u\>!) = \£'\ ■ (-L) + (n - \£'\) ■ {1/q) ■ = -L < 0. Let {a;^}^ be the 
set of all minimal pseudo-codewords. Then u' = agu^^^ for some choice of {a^}^ where 
all ai are non-negative. Therefore, there exists at least one minimal pseudo-codeword, say 
u", such that (cj", A') < 0. (It is clear that u" ^ u.) 

Because u" ^ 0, it is easy to see that (cj", A) ^ {lj",X'). This implies that {lj",X) < 0, 
which is the promised contradiction. 
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